Skip to content

Add InternVL3.5 model support#4613

Open
houleisai wants to merge 1 commit into
PaddlePaddle:developfrom
houleisai:Intervl3.5
Open

Add InternVL3.5 model support#4613
houleisai wants to merge 1 commit into
PaddlePaddle:developfrom
houleisai:Intervl3.5

Conversation

@houleisai

Copy link
Copy Markdown
### PR 类型

新模型支持 / Models

### PR 变更

新增 InternVL3.5-1B 在 PaddleFormers 中的模型接入,包括配置、模型、Processor、ImageProcessor、Auto 注册、单测与验证脚本。

### 主要内容

1. 新增 `paddleformers/transformers/internvl3_5`
   - `InternVLChatConfig`
   - `InternVisionConfig`
   - `InternVLChatModel`
   - `InternVisionModel`
   - `InternVLProcessor`
   - `InternVLImageProcessor`

2. 接入 Auto 入口
   - `AutoConfig`
   - `AutoModel`
   - `AutoProcessor`
   - `AutoImageProcessor`

3. 支持从原始 HuggingFace 权重加载
   - 原始模型路径:`nternVL3_5-1B`
   - 支持 `convert_from_hf=True`
   - 已验证 `model.safetensors` 权重可完整加载

4. 新增测试
   - `tests/transformers/internvl3_5/test_modeling.py`
   - `tests/transformers/internvl3_5/test_processor.py`

### 验证结果

#### 1. 单卡前向精度对齐

使用 FP32 对齐 transformers 与 PaddleFormers 前向 logits:

```text
logits shape: (1, 264, 151936)
logits mean abs diff: 0.00002021
logits p95 abs diff:  0.00006688
logits p99 abs diff:  0.00014514
logits max abs diff:  0.00109911

满足 1e-2 量级要求。

2. 生成结果对齐

使用 greedy search,max_new_tokens=10,与 transformers 生成前 10 个 token 完全一致:

torch tokens:
[1096, 374, 264, 36190, 315, 264, 2766, 1809, 13, 576]

paddle tokens:
[1096, 374, 264, 36190, 315, 264, 2766, 1809, 13, 576]

first10_equal: True

3. GSM8K 300 步训练 loss

loss 整体呈下降趋势,训练过程正常,无 NaN/Inf。

4. 单测结果

../miniconda3/envs/paddleformers/bin/python -m unittest \
  tests.transformers.internvl3_5.test_modeling \
  tests.transformers.internvl3_5.test_processor

结果:

Ran 12 tests in 1.220s
OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant